Resist AI by rethinking assessment

While some academics embrace large language models in higher education, Angelo Pirrone suggests that take-home assessments should be given a marginal role in favour of oral exams and classroom assignments, which mitigate or avoid the risks posed by AI tools such as ChatGP

Towards an atlas of canonical cognitive mechanisms

A central goal in Cognitive Science is understanding the mechanisms that underlie cognition. Here, we contend that Cognitive Science, despite intense multidisciplinary efforts, has furnished surprisingly few mechanistic insights. We attribute this slow mechanistic progress to the fact that cognitive scientists insist on performing underdetermined exercises, deriving overparametrised mechanistic theories of complex behaviours and seeking validation of these theories to the elusive notions of optimality and biological plausibility. We propose that mechanistic progress in Cognitive Science will accelerate once cognitive scientists start focusing on simpler explananda that will enable them to chart an atlas of elementary cognitive operations. Looking forward, the next challenge for Cognitive Science will be to understand how these elementary cognitive processes are pieced together to explain complex behaviour

Is attentional discounting in value-based decision making magnitude sensitive?

Choices in value-based decision making are affected by the magnitude of the alternatives (i.e. the summed values of the options). Magnitude sensitivity has been instrumental in discriminating between computational models of choice. Smith and Krajbich [(2019a). Gaze amplifies value in decision making. Psychological Science, 30(1), 116–128. https://doi.org/10.1177/0956797618810521] have shown that the attentional drift-diffusion model (aDDM) can account for magnitude sensitivity. This is because the discount parameter on the value of the nonfixated alternative ensures faster choices for high-magnitude alternatives, even in the case of high-magnitude equal alternatives compared to low-magnitude equal alternatives. Their result highlights the importance of visual fixations as a mechanism for magnitude sensitivity. This rationale relies on the untested assumption that the discount parameter is constant across magnitude levels. However, the discount parameter could vary as a function of the magnitude of the alternatives in unpredicted ways; this would suggest that the ability of the aDDM to account for magnitude sensitivity has been misinterpreted by previous research. Here, we reanalyse previous datasets and we directly test whether attentional discounting scales with the magnitude of the alternatives. Our analyses show that attentional discounting does not vary with magnitude. This result further strengthens the aDDM and the role that visual fixations could play as an explanation of magnitude sensitivity

Input-dependent noise can explain magnitude-sensitivity in optimal value-based decision-making

Recent work has derived the optimal policy for two-alternative value-based decisions, in which decision-makers compare the subjective expected reward of two alternatives. Under specific task assumptions — such as linear utility, linear cost of time and constant processing noise — the optimal policy is implemented by a diffusion process in which parallel decision thresholds collapse over time as a function of prior knowledge about average reward across trials. This policy predicts that the decision dynamics of each trial are dominated by the difference in value between alternatives and are insensitive to the magnitude of the alternatives (i.e., their summed values). This prediction clashes with empirical evidence showing magnitude-sensitivity even in the case of equal alternatives, and with ecologically plausible accounts of decision making. Previous work has shown that relaxing assumptions about linear utility or linear time cost can give rise to optimal magnitude-sensitive policies. Here we question the assumption of constant processing noise, in favour of input-dependent noise. The neurally plausible assumption of input-dependent noise during evidence accumulation has received strong support from previous experimental and modelling work. We show that including input-dependent noise in the evidence accumulation process results in a magnitude-sensitive optimal policy for value-based decision-making, even in the case of a linear utility function and a linear cost of time, for both single (i.e., isolated) choices and sequences of choices in which decision-makers maximise reward rate. Compared to explanations that rely on non-linear utility functions and/or non-linear cost of time, our proposed account of magnitude-sensitive optimal decision-making provides a parsimonious explanation that bridges the gap between various task assumptions and between various types of decision making

Computational scientific discovery in psychology

Scientific discovery is a driving force for progress, involving creative problem-solving processes to further our understanding of the world. Historically, the process of scientific discovery has been intensive and time-consuming; however, advances in computational power and algorithms have provided an efficient route to make new discoveries. Complex tools using artificial intelligence (AI) can efficiently analyse data as well as generate new hypotheses and theories. Along with AI becoming increasingly prevalent in our daily lives and the services we access, its application to different scientific domains is becoming more widespread. For example, AI has been used for early detection of medical conditions, identifying treatments and vaccines (e.g., against COVID-19), and predicting protein structure. The application of AI in psychological science has started to become popular. AI can assist in new discoveries both as a tool that allows more freedom to scientists to generate new theories, and by making creative discoveries autonomously. Conversely, psychological concepts such as heuristics have refined and improved artificial systems. With such powerful systems, however, there are key ethical and practical issues to consider. This review addresses the current and future directions of computational scientific discovery generally and its applications in psychological science more specifically

Evolving Understandable Cognitive Models

© 2022 The Author(s), published by the Applied Cognitive Science Lab, Penn State. This is the accepted manuscript version of a conference paper which has been published in final form at http://www.frankritter.com/papers/ICCM2022Proceedings.pdfCognitive models for explaining and predicting human performance in experimental settings are often challenging to develop and verify. We describe a process to automatically generate the programs for cognitive models from a user-supplied specification, using genetic programming (GP). We first construct a suitable fitness function, taking into account observed error and reaction times. Then we introduce post-processing techniques to transform the large number of candidate models produced by GP into a smaller set of models, whose diversity can be depicted graphically and can be individually studied through pseudo-code. These techniques are demonstrated on a typical neuro-scientific task, the Delayed Match to Sample Task, with the final set of symbolic models separated into two types, each employing a different attentional strategy

Genetic Programming for Developing Simple Cognitive Models

©2023 The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY), https://creativecommons.org/licenses/by/4.0/Frequently in psychology, simple tasks that are designed to tap a particular feature of cognition are used without considering the other mechanisms that might be at play. For example, the delayed-match-to-sample (DMTS) task is often used to examine short-term memory; however, a number of cognitive mechanisms interact to produce the observed behaviour, such as decision-making and attention processes. As these simple tasks form the basis of more complex psychological experiments and theories, it is critical to understand what strategies might be producing the recorded behaviour. The current paper uses the GEMS methodology, a system that generates models of cognition using genetic programming, and applies it to differing DMTS experimental conditions. We investigate the strategies that participants might be using, while looking at similarities and differences in strategy depending on task variations; in this case, changes to the interval between study and recall affected the strategies used by the generated models

Heuristic Search of Heuristics

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG. All rights are reserved by the Publisher. his is the accepted manuscript version of a conference paper which has been published in final form at https://doi.org/10.1007/978-3-031-47994-6How can we infer the strategies that human participants adopt to carry out a task? One possibility, which we present and discuss here, is to develop a large number of strategies that participants could have adopted, given a cognitive architecture and a set of possible operations. Subsequently, the (often many) strategies that best explain a dataset of interest are highlighted. To generate and select candidate strategies, we use genetic programming, a heuristic search method inspired by evolutionary principles. Specifically, combinations of cognitive operators are evolved and their performance compared against human participants’ performance on a specific task. We apply this methodology to a typical decision-making task, in which human participants were asked to select the brighter of two stimuli. We discover several understandable, psychologically-plausible strategies that offer explanations of participants’ performance. The strengths, applications and challenges of this methodology are discussed